For many applications, an artificial neural network (ANN) architecture can be used as a viable alternative to von Neumann style processor architectures. Indeed, ANN architectures are already used in many applications, e.g., for pattern recognition. ANNs typically comprise interconnected sets of nodes that act like and are often referred to as “neurons.” An example of an ANN is shown in FIG. 1, wherein the neurons are labeled N1, N2, etc. Each neuron has an output, typically referred to as an “axon,” and one or more inputs provided via connections often referred to as “synapses.” As shown in FIG. 1, ANNs may have a plurality of layers, often referred to as input layers (e.g., Layer 1), which receive inputs (e.g., X1, X2, etc.), “hidden” layers (e.g., Layer 2), which receive outputs from and provide outputs to other neurons, and output layers (e.g., Layer 3), which produce outputs (e.g., Y1, Y2, etc.). The synapses (indicated as lines between neurons in FIG. 1) typically have weights or coefficients associated therewith to amplify or otherwise modify the signals carried by the synapses and/or received by the neuron. The neuron typically sums the signals of all “upstream” connected synapses (i.e., synapses that provide inputs to the neuron) and any bias applied to the neuron, and through an activation function changes the output axon, which may then be provided via one or more “downstream” synapses to subsequent neurons or as an output of the ANN (for example, in FIG. 1, neuron N1 is “upstream” of neuron N5). The result is a network that can process incoming signals and drive output signals.
Mathematical representations of ANNs have been implemented in software with success and are used in areas such as object detection, voice recognition, and data mining applications, among others. Software-implemented ANNs can be dynamic in that they can be “trained” to solve many different problems. Software implemented ANNs can be more efficient than traditional coding algorithms, but there is still a significant gap in performance relative to an optimal ANN. A common way of implementing a high performance ANN in a semiconductor technology is to train a specific network for a specific task, and then hard-code that solution directly into the semiconductor technology. While this can produce high computing efficiency for a particular implementation, it also results in the loss of the ability to subsequently reconfigure the network by changing weights, biases, or interconnections between neurons or by adding or removing neurons. Other ANNs use memory elements such as SRAM or volatile registers to create an ANN that can be programmed. However, these memory elements are physically large, limit the feasible size of the ANN, and may also limit the flexibility of the connections between neurons.